Sensors in an environment may be used to infer information about occupants of the environment in real-time. The data provided by sensors in an environment may include a limited number of noisy observations. The sensors may have limited fields of view, and may only be able to provide data about the environment with low granularity. This may make interpretation of the data from the sensors in real-time difficult and result in incorrect inferences about the occupants of the environment. It may be difficult to generate a model that can be used to interpret data from sensors in varying environments that will provide accurate inferences about the occupants of those environments in real-time, as sensor data may need to be interpreted differently for different environments.